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Abstract 

In this work we analyze the solutions of a simple system of coupled phase oscillators 
in which the connectivity is learned dynamically. The model is inspired in the process 
of learning of birdsong by oscine birds. An oscillator acts as the generator of a basic 
rhythm, and drives slave oscillators which are responsible for different motor actions. 
The driving signal arrives to each driven oscillator through two different pathways. One 
of them is a direct pathway. The other one is a reinforcement pathway, through which the 
signal arrives delayed. The coupling coefficients between the driving oscillator and the 
slave ones evolve in time following a Hebbian-like rule. We discuss the conditions under 
which a driven oscillator is capable of learning to lock to the driver. The resulting phase 
difference and connectivity is a function of the delay of the reinforcement. Around some 
specific delays, the system is capable to generate dramatic changes in the phase difference 
between the driver and the driven systems. 

We discuss the dynamical mechanism responsible for this effect, and possible applica- 
tions of this learning scheme. 

1 Introduction 

Biological systems are capable of generating an extremely rich variety of motor commands. 
Most impressively, in many cases, these articulated commands are learned through experience. 
The dynamical processes involved in learning are the focus of extensive research, both in order 
to gain knowledge on how living systems operate, as well as an inspiration for the design of 
artificial systems capable of adaptation and learning. 

In this framework, the acquisition of song by oscine birds is a wonderful animal model for 
the study of how nontrivial behavior can be learned PP, 0. First, it has tight parallels with 
speech acquisition, since birds must hear a tutor during a sensitive period, and practice while 
hearing themselves, in order to learn to vocalize P, Second, the discovery of discrete nuclei 
(large sets of neurons) involved in the process of producing and learning song has provided a 
neural substrate for behavior, turning this animal model in a rich test bench to study the neural 
mechanisms of learning. Finally, recent physical models of birdsong production have provided 
insight on how the activity of different neural populations can be associated to acoustic features 
of song |U. 

In the last years, much has been learned about the neural processes involved in the gener- 
ation of song by oscine birds. As in many other biological systems, song production is based 
on a rhythmic activity. Namely, a syllable is repeated when a motor gesture is performed 
periodically. This gesture involves the coordination of a respiratory pattern and the rhythmic 
activation of the muscles controlling the syrinx (i.e. the avian vocal organ). Recent work 
has unveiled that many important acoustic features of birdsong are in fact determined by the 
phase difference between two basic gestures: the pressure at the air sacs and the tension of 
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the ventral muscles controlling the syrinx 4 . As the result of an extensive research program, 
the specific roles of different neural nuclei in this process are being understood. 

It was through lesions and observation of behavior that the nuclei involved in the generation 
of motor activities responsible for the production of song were identified The so called 
"motor pathway" is constituted by two nuclei called HVC (high vocal center) and RA (robustus 
nucleus of the archistriatum). During the production of song, the HVC sends instructions to 
the RA, which in turn, sends instructions to two different nuclei: the nXIIts, which enervates 
the syringeal muscles, and the RAm, in control of respiration. 

Work on Zebra finches (taeniopygia guttata) has analyzed in detail the exact time relation 
between the firing of neurons in HVC and RA during song [Hj. The picture emerging from 
these experiments is that during each time window in the RA sequence, RA neurons are driven 
by a subpopulation of RA-projecting HVC neurons which are active only during that window 
of time [S] . These experiments suggest that the premotor burst patterns in RA are basically 
driven by the activities of HVC neurons (McCasland et al. 1981). If this is the case, the 
architecture of the connectivity between the HVC nucleus and the RA nucleus determines the 
complex patterns of activity in RA neurons. 

A second pathway thoroughly studied is the anterior forebrain pathway (AFP). This path- 
way connects indirectly the nuclei HVC and RA, as shown in Fig. 1. In contrast with the 




Figure 1: The activity of the nucleus HVC is represented by the phase variable <pi, and drives 
the neurons in nucleus RA through 2 different pathways: a direct way synapses onto the cells 
described by the phase variable </>2, and an indirect projection, controlling the activity of </>3 
of another set of cells. The phase (f>3 is delayed with respect to <\>\ in a. The strength k of the 
connection between (f>2 and 4>i depends on the phase difference 4>2 — 4>i ■ 

motor pathway, this pathway contributes only minimally to the production of song in adults 
[Sj. However, it has been shown that the lesions to the nuclei in this pathway during learning 
profoundly alters the bird's capability of developing normal song [7] . The output of this 
pathway is the lateral magnocellular nucleus of the anterior neostriatum (LMAN). Individual 
RA neurons receive inputs from both LMAN and HVC nucleus, which is consistent with the 
picture that experience related LMAN activity facilitates certain HVC-RA synapses, helping 
to build the neural architecture necessary to produce the adult song. According to this pic- 
ture, a sequence of bursts generated at a RA-projecting HVC will induce some activity in RA, 
and also eventually induce the activity of LMAN that will lead to either the potentiation or 
depression of the connection. This signal, however, requires a processing time through the 
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AFP, which has been estimated in approximately 40 ms in Zebra finches 1 1 . 

If we are interested in the generation of the instructions controlling a motor output, it is 
sensible to model the process in terms of global oscillators. In this way, the periodic activity 
of the HVC nucleus when a syllable is being repeated is represented in terms of the oscillation 
of a simple oscillator. The nucleus RA, being driven by HVC, is represented by a second 
oscillator driven by the first one. In this framework, the dynamics of learning is the dynamics 
of the coupling coefficients between the oscillators. 

With this biological inspiration, we make a computational study of the dynamical mecha- 
nisms by which a driven neural oscillator (representing the activity of a subpopulation of RA 
neurons) can learn to lock to its driver (representing the neurons in the nucleus HVC). The 
forcing that HVC performs upon RA through the indirect pathway AFP has been represented 
by a delayed reinforcement. The learned phase difference between driver and driven oscilla- 
tors has been studied as a function of the reinforcement delay. In certain parameter regimes, 
small changes in the delay have been found to lead to important changes in the learned phase 
difference, and we explain this effect in terms of the dynamics of the system. 

The work is organized as follows. In section 2 we discuss the mathematical model used 
to emulate the activities in our neural circuits. The solutions of this model are discussed 
in section 3. The consequences of the dynamical skeleton in terms of learning dynamics are 
discussed in section 4. Section 5 contains applications of these mechanisms to rate models of 
neural populations. Section 6 presents our conclusions. 



Rhythmic activity play an important role in many neural systems |13j . Cyclic neuronal activity 
is typically modeled in terms of periodic oscillators. Moreover, for those cases where the 
amplitude of the oscillations does not vary, it is possible to further reduce the dynamics of the 
oscillator to a phase variable. Winfree, Kuramoto and others have shown important features 
of coupled systems following this approach ^2] . 

In a recent work, a generalized Kuramoto model of coupled oscillators with a slow cou- 
pling dynamics was investigated |I3| . Inspired by a Hebbian like learning paradigm, the 
authors wrote a dynamical system for the evolution of the coupling which would strengthen 
synchronized states. The system of oscillators presented an interesting dynamics: the original 
difference between the natural frequencies of the oscillators served as a driving force in the 
dynamics of the phase differences between the oscillators. When the coupling parameters fell 
within a given range, the oscillators were eventually able to lock. In addition, a plasticity 
ingredient was incorporated, acting at a slower time scale. Namely, the dynamics of the cou- 
pling strength between the oscillators was driven by the phase difference between them. Once 
again, for some parameter values, the oscillators would end up locked. 

In this work we are interested in the process of locking an oscillator to its driver, exploring 
the dynamical consequences of reinforcing the driving through a second pathway. In order to 
model this effect, three phase variables </>i, <p2 and </>3 are introduced. In terms of the inspiring 
problem, <pi represents the oscillation of a first nucleus generating a periodic instruction (as 
HVC, with its periodic dynamics). The phase <j>2 stands for the activity of some region of the 
nucleus RA, which contains premotor neurons controlling some aspect of the song production. 
This phase is driven by <p\. Finally, <p 3 parametrizes the activity of the indirect pathway, and 
its dynamics is assumed to be the same as that of (f>i, delayed some time r. Modelling the 
activity of HVC as a simple a harmonic function of frequency u\, this delay translates into a 
phase a — lj%t. 

According to these hypothesis, the model describing the dynamics of these variables reads: 
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dt 

dt 
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uj 2 - fcsin(0 2 - 4>i) - k 13 sm(4>2 - <j> 3 ) 



(1) 
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dk 



= 7Cos(</> 2 - 4>\) — k 



(3) 



with (f>3 = <j>i — a. 

In Fig. 1 we show a sketch of the three nuclei and their connections indicating the associated 
dynamical variables. Replacing <ps = <p\ — a in Eq. 2, and scaling the equations, the following 
dynamical system for the phase difference 2 — 4>i = <j) an d k is obtained, 



^ = 1 — k sin — fci3 sin(0 + a) 

^ = e(7cos0 — fc). 
at 



(4) 
(5) 



We are interested in understanding the dynamics of this set of equations, stressing on the 
stationary solutions that the system can "learn" . We are particularly interested in finding out 
whether there are delays that can provide any special advantage in the process of learning to 
control a periodic motor pattern. 

3 Solutions 

For a qualitative understanding of the solutions presented by this system of equations, we can 
compute the nullclincs: curves for which each variable is stationary. They are: 

k = [1 — fc 13 sin(</> + a)]/sin0. (6) 
k — 7cos(0) (7) 

In Fig. 2 we display the nullclines for different values of the parameters. The intersections of 
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Figure 2: Nullclines = (full line) and k — (dashed line), 7 = 2, a = Stt/A and different 
values of fci 3 : (a) fci 3 = 1.5, (b) fci 3 = 1, and (c) fci 3 = 10. 

the nullclines give the fixed points of the system (fco, 0o)- 
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In the parameter range fci3 S [0, 1.42], the nullcline of Eq. 10 presents two branches: the 
first one with a minimum, the second one with a maximum. Depending on the parameter 
values, one of the branches or both might intersect the nullcline of Eq.Ql. These intersections, 
when they occur, lead to the appearance of a saddle and a node in a saddle node bifurcation. 
For 7 sufficiently large, two attracting fixed points (separated by two saddles) can coexist. On 
the other hand, for 7 sufficiently small, there are no intersections between the nullclincs and 
therefore, no stationary phase difference between the driver and the driven oscillator can be 
established. 

The topological organization of the nullclines present qualitative changes as the system 
parameters are varied. These changes leave their imDrint in the deoendence of the stable fixed 
point angula the fixed 




Figure 3: Stationary solutions for as a function of a for 7=1. Results for (a) fci3 = 0.9, 
(b) fci3 = 1.5, (c) fci3 = 1.8 and (d) fci3 = 15. In all the cases the solid lines correspond to 
stable solutions and the crosses indicate the branches of unstable solutions. 

points as a function of the reinforcement delay a, varied between (0, 2ir). The different insets 
correspond to a different value of the reinforcement parameter ^13. In the figure, the solid 
lines indicate the linearly stable solutions and the lines with crosses the unstable ones (which 
in all cases are saddle points). For small values of the coupling, there are delays for which 
no fixed points exist (Fig. 3a). At specific delays, stable and unstable fixed points are born 
in saddle node bifurcations. In terms of nullclines, this corresponds to situations in which 
the second nullcline of Eq. © presents a minimum which touches the nullcline of Eq. J7J) (see 
Fig 2. a). As the reinforcement strength is increased, the region of reinforcement delays for 
which no stationary solutions exist decreases in size. As the reinforcement parameter kis is 
further increased, the regions with no solutions disappear and the bifurcation curves meet at a 
transcritic point (Fig. 3b) around which, a narrow zone of multistability appears. Finally, as 
the reinforcement strength is further increased, the angular position of the fixed points varies 
monotonically (Figs. 3c, 3d). 

The existence of these bifurcation curves has profound consequences in terms of learning. 
Notice that close to certain values of the delay a, a minimal change in a gives rise to important 
differences in the equilibrium phases learned by the system. In the following section, we will 
discuss potential consequences of this bifurcating structure in a learning process. 
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4 Interpretation 



The animal model inspiring our dynamical model is the motor pathway in oscine birds. Part 
of this pathway is the nucleus RA containing excitatory neurons, some of which enervate 
respiratory nuclei, and others enervate the nucleus nXIIts, that projects to the muscles in the 
syrinx ^3]. These two populations are segregated into different regions of the RA structure. 

Recently, the study of the avian vocal organ allowed us to associate acoustical features 
of the song with properties of the muscle instructions necessary to generate the song. The 
production of repetitive syllables requires a cyclic expiratory gesture, and a cyclic gesture of the 
syringeal muscles |15j . Sound is produced by labia located at the junction between bronchii 
and tract, obstructing periodically the airflow . The model mentioned above describes the 
departure of the midpoint of a labium from the prephonatory position, x 0J|S]: 



where e(t) is a function of the activity of ventral muscles, whereas B{t) is a function of the 
bronchial pressure. This model has been tested, by using experimentally recorded e(t) and 
B(t). The resulting x(t) was remarkably similar to the one produced while the physiological 
data had been recorded [Hj- 

The phase difference between the e(t) and B(t), responsible for important acoustic features 
of song (such as the dynamics of the syllabic fundamental frequency) originates in RA. Recent 
work has unveiled that direct connections between respiratory nuclei and nXIIts can affect 
the final value of the phase difference. Yet, it is at RA that the neurons driven by HVC also 
receive input from the indirect pathway AFP, and therefore it is at this level that the phase 
difference between gestures can be altered. 

In order to apply the results of the previous section to a learning strategy for birdsong, 
let us assume that two oscillators represent the cyclic activity of RA during the production 
of song. One of the oscillators mimics the activity of the neural population enervating the 
nXIIts nucleus, while the other oscillator represents the population of neurons which control 
the respiratory pattern. Both oscillators are assumed to be driven by the nucleus HVC, 
presenting a global activity with a syllabic rhythm. The reinforcement oscillator drives both 
RA oscillators with a signal equal to that of HVC, but delayed in a phase a. Figure 4a 
illustrates the proposed architecture. 

According to Fig. 3, if the two oscillators representing RA are strongly reinforced by the 
AFP circuit (i.e., if fci3 is large as in Fig. 3d), for every value of the forcing there will be a locked 
state. The phase difference between their own oscillation and the one of the driver will be the 
same for both oscillators, whatever the value of the delay a. However, a different situation 
is found if each one of the oscillators is reinforced through a different coupling strength fci3. 
Imagine that for one of them, fci3 is similar to the one used to generate Fig. 3b, while the other 
one is forced though a coupling strength as the one used to generate Fig. 3d. In this case, 
depending on the value of the delay, qualitatively different phase differences can be achieved 
between each RA oscillators and the driver (and therefore, between the two RA oscillators 
themselves) . 

In Fig. 4b, the value of the phase difference A</>o is displayed as function of a for different 
pairs of oscillators. We have fixed 7=1. In all cases, one of the oscillators, which is taken 
as a reference, is assumed to be coupled to the AFP circuit through the parameter fci3 = 1.5, 
corresponding to the situation of Fig. 3b. For the other oscillator, we consider different values 
of fci3 > 1.5. All these couplings give rise to stationary solutions 4> that decrease monotonously 
as a function of a (see Figs. 3c and 3d). The curves in Fig. 4b are the difference between the 
stationary solution of both oscillators, for several values of the second coupling constant. In 
order to indicate in detail how this difference is computed we take the example of the solid 
line in Fig. 4b, for which the second oscillator is coupled to the AFP circuit with a strength 
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Figure 4: (a) Two oscillators representing neuron populations (nXIIts and RAm) driven 
by a master oscillator (HVC) will learn to follow the master at different delays. The two 
populations control different aspects of the song production apparatus: the syringeal nucleus 
and the respiratory muscles, respectively, (b) Phase difference A(/>o as a function of a. The 
solid line corresponds to the difference between the stationary solutions with ki3 = 15 and 
/ci3 — 1.5; the dashed line to &13 = 5 and /C13 = 1.5; the dotted line to fci3 = 2.5 and ki3 — 1.5; 
and the dashed-dotted line to £13 = 1.8 and /C13 = 1.5. All calculations are performed with 
7=1. 

&13 = 15. This curve corresponds to the difference of the solutions indicated with I and II in 
Fig. 3b (i.e. the stationary solution for the reference oscillator) and the curve III on Fig. 3d 
(the stable solution for the second oscillator). We notice, however, that there is a small range 
of a for which the branches of solutions I and II coexist. For such values of a we have taken 
branch II as the solution for the reference oscillator. This choice leads to the lowest possible 
value of phase difference between the two oscillators, since branch III is closer to branch II 
than to branch I. (Hence, the phase difference could be even higher than the one shown in Fig. 
4b). It should be stressed that there are delays (close to the value a = 37r/4 for the parameters 
used here) for which small displacements can lead to a huge change in the stationary phase 
difference between the oscillators. Notice that Fig. 4b is 7r-periodic, so there is a similar effect 
around a = 7it /A. 

The results in Fig. 4b, are robust with respect to changes in the parameter 7. We have 
checked that for all 7 e (0.5, 10), the transcritical bifurcation occurs at a « 37r/4 and a « 77r/4. 
Moreover, whenever a is in the neighborhood of any of these critical values, the phase difference 
between oscillators with different /C13 is highly sensitive to the value of a. Yet, the values of £^3 
relevant for observing the mentioned phenomena do depend on 7. For instance, the value of 
fci3 at which the transcritical bifurcation is observed increases with 7. It goes from fc 13 ~ 1.2 
for 7 = 0.5 to &13 ~ 6 for 7 = 10. 

Figure 5 illustrates how different the learned syllables can be for small changes in the 
reinforcement delay, if they occur around a — 3n/A. The figure displays a sonogram showing 
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Figure 5: The sonograms of two songs which consist of a repetition of the same syllable, for 
which the phase difference learned is (a) A0o = and (b) A0o = tt/2. The parameters of Eqs. 
El and El are e(t) = 7 10 7 + 6 10 7 cos(27rf/44100) a .u, B(t) = 5 10 2 + 10 3 cos(27rf/44I00 + A0 O ) 
a.u, C = 2 10 9 a.u. 



the time evolution of the fundamental frequency of the sound produced by the model of the 
syrinx, when driven by the "learned" phase differences. 

5 Application to rate models 

The Kuramoto model describing the time evolution of phase differences between oscillators 
constitutes a popular model, particularly suited for analytic work. Yet, we explored whether 
these effects are also present in other models. We tested the basic findings of the previous 
sections in rate models for the activities of neural populations. 

Rate models are introduced to account for the dynamics of the average activity of neural 
nuclei. For a problem involving a macroscopic motor control program it is a suitable level of 
description. In particular, a widely used model for the average activity of two subpopulations 
(exitatory and inhibitory) is the Wilson Cowan system of equations ^Hj • 

We let x and y stand for the activities of the excitatory and inhibitory subpopulations 
respectively. In Jl] it was shown that in the portions of RA with neurons projecting to Xllts 
and to respiratory nuclei, the (excitatory) projecting neurons coexist with inhibitory neurons. 
We drove the equations ruling their dynamics with two signals. One represents HVC activity 
and the other one, the activity in nucleus LMAN, assumed to be a delayed copy of the first 
signal. A typical hebbian rule was used to describe the dynamics of the coupling between HVC 
and RA. The system reads 

—x + S[p x + ax + by + k cos(wt) + ki 3 cos(w< — a)] 
-y + S(p y + cx + dy) (10) 
Ace cos{u}t) — k 
with a saturation function S{x) = 1/(1 + e~ x ). 

We integrated these equations for the whole range of delays a € [0, 2tt). In Fig. 6 we show 
a window representing a bifurcation around a = 297r/15, where two qualitatively different 
period-one solutions can be found. The transition occurs through a bubble in parameter space 
where a period-two exists. It is interesting that the different period-one solutions (illustrated in 
the insets), are locked to the periodic forcing frequency at different phases. As k%s is increased, 
only a single period-one solution is present. Therefore, two subpopulations reinforced through 
a delay a around this bifurcating value with these weights will lock at a phase difference within 
a wide range of values as the delay is slightly increased. 
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Figure 6: Bifurcation diagram for the Wilson-Cowan system (Eq. 10) forced with frequency 
u = 0.3. The horizontal axis represents the delay a and the vertical axis represents the 
maximum amplitude A of the oscillations in phase space. A broad phase difference between 
two slave oscillators with weights = 2 and &13 = 3 can be achieved within a narrow range 
of delays 297r/15 < a < 3l7r/16. On the insets, solutions x(t) along with forcing function 
cos{ujt) for the three different regimes found. The equations were integrated with the following 
parameters: p x — —5.75, p y — — 1, a = 10, b = —1.5, c = 2, d = 2 and A = 68. 

6 Conclusions 

In a recent work J3| , the process of learning a phase difference between two phase oscillators 
coupled with a slow varying coupling constant was described. Here, the impact of subjecting 
the driven system to two delayed copies of a signal is studied. This design is inspired in an 
animal model, oscine birds, which learn their song by modifying the architecture of connections 
within the motor pathway. Our model is a caricature of the complex processes that have been 
described for the animal model, which involved the convergence to a motor nucleus of two 
signals, separated by a delay. Our study indicates that for some values of the amplitude of the 
reinforcement, the learned phase difference between driver and slave depends strongly on the 
delay of the reinforcement. This allows to conceive simple strategies of learning motor gestures 
that require a tuning between different neural populations. We precisely illustrate the power 
of this strategy with an example of the inspiring biological problem. We show that playing 
with small variations in the delay of the reinforcement, completely different syllables in a bird 
song can be generated. 

In the problem of bird song learning, the timing of the indirect pathway (the anterior 
forebrain pathway) can be changed |17j by dopaminergic input. Not much is known about 
the precise nature of the coding used by a bird to translate an error into the indirect signal. 
Measures of activity in the nucleus LMAN (the last one of the AFP pathway, which projects 
onto RA) in a juvenile bird learning his song are still not possible. The simple model presented 
in this work allows us to explore theoretically the dynamical mechanisms that enter into a 
learning scheme compatible with the basic ingredients present in the animal model. Moreover, 
it provides a new control mechanism applicable to artificially designed neural systems. 
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